Summary

Resolves: #676
Resolves: #900
Resolves: #926
Resolves: #1531
Resolves: #1554
Resolves: #1728
Resolves: #1854
Resolves: #1889
Resolves: #1904

This PR is very dense but I think this solves a lot of our playback quirks! The crux of the issue is that we have two sources of truth for track selection. Jellyfin and the actual files read via VLCKit.

• Jellyfin handles this as External Audio > External Subtitles > Video > Audio > Subtitles.
• VLC handles this as File Track Order > External Audio > External Subtitles.

Previously, we were resolving this by just moving tracks around at the beginning. This ended up causing issues for us with the video player rewrite as we have a kind of 'sidecar' system for external files now. So, when remapping indexes, we didn't know the order of these files but just a guess. On top of this, we were not accounting for files types that Jellyfin was hiding in the MediaSource but exist in the file. Particularly, when Text or Image subtitles types are disabled. This then causes issues as our index we are using in Swiftfin doesn't match anything in Jellyfin anymore. To work around this, I build an indexMap with local and server indexes that are built first based on the Jellyfin provided MediaSources that are then revised as the 'sidecar' subtitles and other information load in. Round 1 is used to get the default / selected track for playback and Round 2 sures these up in case there are issues like Image or Text subtitles being hidden or things like external audio existing in the MediaSources but being unusable to us. Transcoding also causes issues as all internal subtitles because external and move down the row.

The other issue with this is that we can request stream based on default tracks. So, if the default track is AAC and the user switches to MLP / Atmos, a transcode will not be requested because we were given the raw file and we are changing tracks locally. This ave the impression that Swiftfin was not transcoding at all. I've updates this so, as tracks change, audio always requests a new stream to trigger a transcode if needed. Subtitles only triggers a new stream if we are requesting a PGS / Image Subtitle in an external format as those cannot be handled locally and need to be encoded into the video. Documentation and Device Profiles have been updated to reflect this!

Finally, we cannot play external PGS or image subtitles saves as .sup file. These need to be encoded into the file or embedded into the container. For this reason, if I have compatibility mode set to DirectPlay only, I hide these options from the drop down to prevent issues.

I've fully tested every variation and have ZERO issues or mis-selections. The only issue is external audio tracks do not play but the have never played in Swiftfin. I left a TODO to figure this out but basically, these external audio tracks can DirectPlay so Swiftfin doesn't request a transcode but Jellyfin doesn't provide a delivery URL so we need to encode these into the stream to play this.

Testing files with subtitles can be found below. 25mb was the GitHub limit so I had to break this into 2 batches:

Tests 1-3.zip
Tests 4-7.zip

Happy to discuss any of what I've done here! It's all functional and clean (in my eyes) but I've been looking at this too long so any cleanup I can do to make this more universally understandable is welcome feedback!

Videos:

Hopefully this shows what I mean but this is transcodes & directplays with various subtitles counts and audio counts all working without issue. All from a test file with the exception of the MLP / TrueHD which is LOTR since that's like my only file I left that in.

MLP / TrueHD Transcoding (AC3 -> TrueHD)

1.4.1:

If my starting track does not require transcoding, and then I switched to a track that does require transcoding, VLC just switches the track inside of the file, resulting in no playback.

If my starting track does require transcoding, or transcoding occurs at any point, I am only shown the audio track that is included with the new stream. Additionally, the subtitle indexes are incorrect, so you cannot select subtitles while transcoding.

All tracks are always shown to you and are always selectable. If you select a track that requires transcoding a transcode occurs, but if you select a track that does not require transcoding, it will just switch within the file itself. The position of playback is preserved.

External PGS File Transcoding (Text Embedded Subtitles Only)

Transcoding File Test (Text Embedded Subtitles Only)

Transcoding File Test (Text Embedded Subtitles Only & DirectPlay Only)

Transcoding File Test (Image Embedded Subtitles Only & DirectPlay Only)